The normal human visual system provides two separate views of the world through our two eyes. Each eye has a horizontal field of view of about 60 degrees on the nasal side and 90 degrees on the temporal side. A person with two eyes, not only has an overall broader field of view, but also has two slightly different images formed at her two retinas, thus forming different viewing perspectives. In normal human binocular vision, the disparity between the two views of each object is used as a cue by the human brain to derive the relative depth between objects. This derivation is accomplished by comparing the relative horizontal displacement of corresponding objects in the two images.
Stereoscopic displays are designed to provide the visual system with the horizontal disparity cue by displaying a different image to each eye. Known stereoscopic displays typically display a different image to each of the observers' two eyes by separating them in time, wavelength or space. These systems include using liquid crystal shutters to separate the two images in time, lenticular screens, barrier screens or auto-stereoscopic projection to separate the two images in space, and the use of color filters or polarizers to separate the two images based on optical properties.
Unfortunately, the visual information provided by a stereoscopic display differs in many ways from viewing real world objects. This difference in visual information or visual cues often leads to user discomfort or the inability to fuse the two images (i.e., to perceive the two images displayed to the two eyes as one fused image rather than two separate images). One of the important differences between the visual information provided by most prior art stereoscopic display systems and the real world environment arises from the fact that the user of these displays must accommodate, or focus, at a single plane in space when viewing a stereoscopic display while the disparity cues that are provided indicate that the objects are at different planes in space. This differs from the real world visual environment where the accommodative and disparity cues provide consistent information. This is a very important difference since the vergence of our eyes and their accommodation distance are reflexively linked, often causing accommodation distance to track convergence distance as the two eyes converge to place an important object onto the fovea of each eye.
Another important difference is that stereoscopic display systems have the ability to augment the disparity cues, providing larger amounts of disparity information than actually exists in the real world. This provision can provide a greater sense of depth and greater perceived separation of objects in an image. This can be particularly important in detection or identification tasks, where it is important to separate a target (e.g., a cancerous lesion, a surveillance threat, or a potential rescue victim) from a complex background. Unfortunately, when this disparity becomes too large, users of stereoscopic display systems often experience discomfort.
It is well understood that there is an upper limit for how much disparity the human brain can fuse (see Binocular Vision and Stereopsis, Chapter entitled “The limits of stereoscopic vision” by Howard and Rogers, p. 159, 1995). It is also well understood that some individuals are not able to interpret stereoscopic information and therefore are unable to benefit from the cues provided by a stereoscopic display system. It is understood to a lesser degree, however, that there is significant individual variability in observers' ability to comfortably fuse two images that have horizontal disparity.
In the field of psychology, it has been demonstrated that the fusional range that is common to all users is much smaller than that for some individual users (see ref. “Image distortions in stereoscopic video systems” Proceedings of the SPIE, Volume 1915, Stereoscopic Displays and Applications IV, by Andrew Woods, Tom Docherty, and Rolf Koch, pp. 1-13, 1993). This research finding is supported by research in the field of optometry, where a method of quantifying the range of convergent and divergent angles that each individual is able to fuse has been applied to understand an individual's visual performance. It is well understood that this range of convergent and divergent angles that can be fused, referred to as an individual's fusional reserve, differs from individual to individual and differs with differences in the accommodative stimulus that is provided. Importantly, research in this field has shown that some individuals can fuse a large range of convergent angles and a small range of divergent angles while other individuals can fuse larger ranges of divergent angles than convergent angles. Because of this individual variability, if a stereoscopic system is developed to provide comfortable stereoscopic images to all users, the range of disparity that can be shown will be extremely limited.
In the field of stereoscopic display it has been realized that a stereoscopic display system cannot provide high quality, comfortable horizontal disparity information unless the system takes into account the limitation set by the human binocular system. U.S. Pat. No. 4,647,965 describes a stereoscopic imaging system that has the ability to adjust the horizontal positions of the corresponding objects in the stereoscopic pair to provide quality and viewing comfort for viewing three-dimensional content. The system consists of image capture, image digitization, image processing, and digital to analog conversion. The image-processing algorithm identifies corresponding objects in the stereoscopic pair, and shifts the objects in one of the two images to reduce the disparities between the views. However, this system does not provide a means for determining an individual's ability or for clustering an individual into a group of individuals that may have similar visual performance. Further, image display as described in this display is performed through analog media that can not be altered for each individual and therefore the images produced by this system cannot be rendered for users of different ability and therefore to be successful this system must reduce the maximum disparity to the small range that all observers can comfortably fuse or accept the fact that some percentage of users will experience discomfort while viewing these stereoscopic images.
U.S. Pat. No. 6,175,379B1 discusses a computer graphics driven stereoscopic display system which also provides a customized rendering for different viewing conditions. Specifically, the system alters the rendering camera separation, focal length, and/or distance from objects in the scene as a function of the viewing conditions that include the display size, viewing distance, and resolution. The intent of this patent is to render stereoscopic information that can be comfortably viewed by all users. As described in this patent, computer graphics experts decide on the acceptable range of disparities that can be fused by all users when rendering the computer graphics scene. Image content outside the fusional area are clipped from view or their contrast and/or focus are reduced to make them unusable. Once again, this system does not provide a means for determining an individual's ability and the output of this system uses analog media that cannot be altered for each individual user who has a different ability.
User comfort has also been considered in commercial products such as the NVIDIA 3D stereo driver. This driver can generate and display a stereoscopic image pair from one computer generated image with embedded depth information about the computer generated graphic objects. This driver comes with a number of controls for use in stereo game playing, including stereo separation adjustment to suit individual adaptation to stereo viewing, hot keys for in-game stereo control, stereo test process for determining the best screen mode to use, and custom game configurations that tailor the stereo process to a wide range of games. This product provides the technical capability to adjust the range of disparity for a range of stereoscopic scenes. However, this system does not provide a method to link the capability of individual users to fuse stereoscopic images to the stereoscopic rendering process. As a result, the viewing comfort can only be achieved by the dynamic adjustment during viewing. Considering that in a game-playing environment the range of disparity may vary significantly from scene to scene, the dynamic nature of the adjustment can be a challenging task for the users. Furthermore, the optimization between viewing comfort and good stereoscopic perception cannot be achieved by a method of dynamic adjustment such as the one provided by NVIDIA as it does not consider the actual range of disparities present in any scene.
There is a need, therefore, for creating an adaptive stereoscopic display system that can obtain and utilize the knowledge of the stereo capability of the user to enhance the rendering of stereo images.